Author Details

Multiclass classification is a fundamental problem for many speech recognition systems. Spoken digit recognition is a multiclass problem of 10 classes. Present paper using Support Vector Machine (SVM) and K-Nearest-Neighbour (KNN) and Ensemble method i.e., Random Forest (RF) to English digit classification. Caffe speech dataset of 2400 input instances (15 speakers*16 repetitions*10 digits) used for experiments. Mel Frequency Cepstral Coefficients (MFCC) features are formed for all input instances. The dataset is divided into training set and testing set with 10%, 30% and 50% of dataset as testing set. Confusion matrices generated with all test cases for all classification methods. Performance of Ensemble method is high compared to SVM and KNN at different number of frames. The highest accuracy achieved by RF method is 97.50% by taking 10% testing data.

Keywords

Caffe, Ensemble Methods, KNN, MFCC, Random Forest (RF), Spoken English Digit, SVM.

Full Text

References

Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, “Caffe: Convolutional architecture for fast feature embedding,” Proceedings of the 22^nd ACM International Conference on Multimedia, pp. 675-678, ACM, 2014.

I. A. Lawal, “Spoken character classification using abductive network,” International Journal of Speech Technology, vol. 20, no. 4, pp. 881-890, 2017.

G. Muhammad, Y. A. Alotaibi, and M. N. Huda, “Automatic speech recognition for Bangla digits,” 2009 12^th International Conference on Computers and Information Technology (ICCIT’09), IEEE, 2009.

Z. Ali, A. W. Abbas, T. M. Thasleema, B. Uddin, T. Raaz, and S. A. R. Abid, “Database development and automatic speech recognition of isolated Pashto spoken digits using MFCC and K-NN,” International Journal of Speech Technology, vol. 18, no. 2, pp. 271-275, 2015.

D. F. Silva, V. M. A. de Souza, G. E. A. P. A. Batista, and R. Giusti, “Spoken digit recognition in Portuguese using line spectral frequencies,” Ibero-American Conference on Artificial Intelligence, Springer, Berlin, Heidelberg, 2012.

I. Bazzi, and D. Katabi, “Using support vector machines for spoken digit recognition,” Sixth International Conference on Spoken Language Processing, 2000.

J. V. Doremalen, and L. Boves, “Spoken digit recognition using a hierarchical temporal memory,” Ninth Annual Conference of the International Speech Communication Association, 2008.

F. Diaz, J. M Ferrández, P. Gomez, and V. Rodellar, “Spoken-digit recognition using self-organizing maps with perceptual pre-processing,” International Work-Conference on Artificial Neural Networks, Springer, Berlin, Heidelberg, 1997.

T. Kitamura, S. Ando, and E. Hayahara, “Speaker-independent spoken digit recognition in noisy environments using dynamic spectral features and neural networks,” Second International Conference on Spoken Language Processing, 1992.

N. Hammami, and M. Sellam, “Tree distribution classifier for automatic spoken Arabic digit recognition,” 2009 International Conference for Internet Technology and Secured Transactions (ICITST 2009), IEEE, 2009.

B. Logan, “Mel frequency cepstral coefficients for music modeling,” ISMIR, vol. 270, 2000.

A. Geron, Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, O’Reilly Media, Inc, 2017.

N. Scaringella, G. Zoia, and D. Mlynek, “Automatic genre classification of music content: A survey,” IEEE Signal Processing Magazine, vol. 23, no. 2, pp. 133-141, March 2006.

C. J. C. Burges, “A tutorial on support vector machines for pattern recognition,” Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 121-167, 1998.

V. Vapnik, The Nature of Statistical Learning Theory, Springer Science and Business Media, 2013.

B. Scholkopf, C. J. C. Burges, and A. J. Smola, “Advances in kernel methods: Support vector machines,” 1998.

Z. Jan, M. Abrar, S. Bashir, and A. M. Mirza, “Seasonal to inter-annual climate prediction using data mining KNN technique,” International Multi Topic Conference, Springer, Berlin, Heidelberg, 2008.

L.-Y. Hu, and M.-W. Huang “The distance function effect on k-nearest neighbor classification for medical datasets,” SpringerPlus, vol. 5, no. 1, p. 1304, 2016.

L. Breiman, “Bagging predictors,” Machine Learning, vol. 24, no. 2, pp. 123-140, 1996.

M. Khondoker, R. Dobson, C. Skirrow, A. Simmons, and D. Stahl, “A comparison of machine learning methods for classification using simulation with multiple real data examples from mental health studies,” Statistical Methods in Medical Research, vol. 25, no. 5, pp. 1804-1823, 2016.

Clustering of Hand Written Digits Using K-Means Algorithm and Self Organizing Maps

Abstract Views :257 | PDF Views:0

Authors

Maddimsetti Srinivas ¹, M. Venkata Srinu ¹, G. L. P. Ashok ¹

Affiliations
1 Koneru Lakshmaiah Education Foundation, Vijayawada, Guntur(Dt.), Andhra Pradesh, IN

Source

International Journal of Research in Signal Processing, Computing & Communication System Design, Vol 4, No 2 (2018), Pagination: 28-32

Abstract

Present work focuses on clustering of MNIST dataset using K-means clustering and Self-Organizing Maps (SOM). Histograms of Oriented Gradients (HOG) descriptors are used to extract the feature vectors and Principal Component Analysis (PCA) is applied on feature vectors to reduce the dimensionality. First two principal components are taken for cluster formation. Purity of cluster metric is used to evaluate the clusters. External criteria with prior information of true class is chosen to validate cluster. The performance of SOM is better than K-means in forming clusters. Out of 10 clusters K-means algorithm missed clusters of 3 digits (0, 7 and 9) whereas SOM missed clusters of 2 digits (5, 9).

Keywords

Clustering, Histograms of Oriented Gradients (HOG), K-Means Clustering, MNIST, Principal Component Analysis, Self Organizing Maps, Unsupervised Learning.

Full Text

References

Y. LeCun, L. D. Jackel, L. Bottou, C. Cortes, J. S. Denker, …. V. Vapnik, “Learning algorithms for classification: A comparison on handwritten digit recognition,” in J. H. Oh, C. Kwon, and S. Cho (eds.), Neural Networks: The Statistical Mechanics Perspective, pp. 261-276, World Scientific, 1995.

F. Lauer, C. Y. Suen, and G. Bloch, “A trainable feature extractor for handwritten digit recognition,” Pattern Recognition, vol. 40, no. 6, pp. 1816-1824, 2007.

Y. Lee, “Handwritten digit recognition using K nearest-neighbor, radial-basis function, and backpropagation neural networks,” Neural Computation, vol. 3, no. 3, pp. 440-449, 1991.

C.-L. Liu, K. Nakashima, H. Sako, and H. Fujisawa, “Handwritten digit recognition: Benchmarking of state-of-the-art techniques,” Pattern Recognition, vol. 36, no. 10, pp. 2271-2285, 2003.

Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” NIPS Workshop on Deep Learning and Unsupervised Feature Learning, vol. 2011, no. 2, 2011.

M. Iwayama, and T. Takenobu, “Cluster-based text categorization: A comparison of category search strategies,” Proceedings of the 18^th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, 1995.

H. Frigui, and R. Krishnapuram, “A robust competitive clustering algorithm with applications in computer vision,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 450-465, 1999.

S. K. Bhatia, and J. S. Deogun, “Conceptual clustering in information retrieval,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 28, no. 3, pp. 427-436, 1998.

J. Hu, B. K. Ray, and M. Singh, “Statistical methods for automated generation of service engagement staffing plans,” IBM Journal of Research and Development, vol. 51, no. 3.4, pp. 281-293, 2007.

S. P. Hung, P. Baldi, and G. W. Hatfield, “Global gene expression profiling in Escherichia coli K12. The effects of leucine-responsive regulatory protein,” Journal of Biological Chemistry, vol. 277, no. 43, pp. 40309-40323, 2002.

D. C. Cirean, U. Meier, L. M. Gambardella, and J. Schmidhuber, “Deep big, simple neural nets for handwritten digit recognition,” Neural Computation, vol. 22, no. 12, pp. 3207-3220, 2010.

A. K. Jain, “Data Clustering: 50 years beyond K-means,” Pattern Recognition Letters, vol. 31, no. 8, pp. 651-666, 2010.

C. Ding, and H. Xiaofeng, “K-means clustering via principal component analysis,” Proceedings of the Twenty-First International Conference on Machine Learning, ACM, 2004.

A. K. Jain, M. N. Murty, and P. J. Flynn, “Data clustering: A review,” ACM Computing Surveys (CSUR), vol. 31, no. 3, pp. 264-323, 1999.

M. Hassaballah, A. A. Abdelmgeid, and H. A. Alshazly, “Image features detection, description and matching,” Image Feature Detectors and Descriptors, pp. 11-45, Springer, Cham, 2016.

I. T. Jolliffe, “Principal component analysis and factor analysis,” Principal Component Analysis, Springer, New York, NY, pp. 115-128, 1986.

N. Dalal, and B. Triggs, “Histograms of oriented gradients for human detection,” International Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 886-893, IEEE Computer Society, 2005.

MNIST. Available: http://yann.lecun.com/exdb/mnist/

T. Kohonen, “The self-organizing map,” Neurocomputing, vol. 21, no. 1-3, pp. 1-6, 1998.

J. Vesanto, and E. Alhoniemi, “Clustering of the self-organizing map,” IEEE Transactions on Neural Networks, vol. 11, no. 3, pp. 586-600, 2000.

D. Arthur, and S. Vassilvitskii, “k-means++: The advantages of careful seeding,” Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, Society for Industrial and Applied Mathematics, 2007.

A. Gron, Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, O’Reilly Media, Inc., 2017.

Username
Password
Remember me